Stream-processing platforms such as Apache Kafka (Kafka™ is a registered trademark of the Apache Software Foundation) may be used to generate streams of messages identified by a number of topics. As a result, producers of the messages may publish information to the topics, and consumers of the information may subscribe to the topics and process the corresponding message streams. Each topic may additionally be divided into multiple partitions, with each partition containing a commit log of an ordered, immutable sequence of messages.
On the other hand, writes to an overburdened partition cannot be redirected to a new physical location for the partition without replicating all data in the partition from the old physical location to the new physical location. Because the replication process may further increase the load on the partition, the old physical location may drop requests to the partition until the replication is complete, which may negatively impact the performance of the partition.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.